System decoder for training accelerators

ABSTRACT

There is disclosed an example of an artificial intelligence (AI) system, including: a first hardware platform; a fabric interface configured to communicatively couple the first hardware platform to a second hardware platform; a processor hosted on the first hardware platform and programmed to operate on an AI problem; and a first training accelerator, including: an accelerator hardware; a platform inter-chip link (ICL) configured to communicatively couple the first training accelerator to a second training accelerator on the first hardware platform without aid of the processor; a fabric ICL to communicatively couple the first training accelerator to a third training accelerator on a second hardware platform without aid of the processor; and a system decoder configured to operate the fabric ICL and platform ICL to share data of the accelerator hardware between the first training accelerator and second and third training accelerators without aid of the processor.

FIELD OF THE SPECIFICATION

This disclosure relates in general to the field of network computing,and more particularly, though not exclusively, to a system and methodfor a system decoder for training accelerators.

BACKGROUND

In some modern data centers, the function of a device or appliance maynot be tied to a specific, fixed hardware configuration. Rather,processing, memory, storage, and accelerator functions may in some casesbe aggregated from different locations to form a virtual “compositenode.” A contemporary network may include a data center hosting a largenumber of generic hardware server devices, contained in a server rackfor example, and controlled by a hypervisor. Each hardware device mayrun one or more instances of a virtual device, such as a workload serveror virtual desktop.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is best understood from the following detaileddescription when read with the accompanying figures. It is emphasizedthat, in accordance with the standard practice in the industry, variousfeatures are not necessarily drawn to scale, and are used forillustration purposes only. Where a scale is shown, explicitly orimplicitly, it provides only one illustrative example. In otherembodiments, the dimensions of the various features may be arbitrarilyincreased or reduced for clarity of discussion.

FIG. 1 is a block diagram of selected components of a data center withnetwork connectivity, according to one or more examples of the presentapplication.

FIG. 2 is a block diagram of selected components of an end-usercomputing device, according to one or more examples of the presentspecification.

FIG. 3 is a block diagram of components of a computing platform,according to one or more examples of the present specification.

FIG. 4 is a block diagram of an AI system including two platforms,according to one or more examples of the present specification.

FIG. 5 is a block diagram of a system decoder, according to one or moreexamples of the present specification.

FIG. 6 is a signal flow diagram illustrating an example broadcastmessage, according to one or more examples of the present specification.

FIG. 7 is a flowchart of a method performed by a system decoder or otherappropriate device, according to one or more examples of the presentspecification.

EMBODIMENTS OF THE DISCLOSURE

The following disclosure provides many different embodiments, orexamples, for implementing different features of the present disclosure.Specific examples of components and arrangements are described below tosimplify the present disclosure. These are, of course, merely examplesand are not intended to be limiting. Further, the present disclosure mayrepeat reference numerals and/or letters in the various examples. Thisrepetition is for the purpose of simplicity and clarity and does not initself dictate a relationship between the various embodiments and/orconfigurations discussed. Different embodiments may have differentadvantages, and no particular advantage is necessarily required of anyembodiment.

A contemporary computing platform, such as a hardware platform providedby Intel® or similar, may include a capability for monitoring deviceperformance and making decisions about resource provisioning. Forexample, in a large data center such as may be provided by a cloudservice provider (CSP), the hardware platform may include rackmountedservers with compute resources such as processors, memory, storagepools, accelerators, and other similar resources. As used herein, “cloudcomputing” includes network-connected computing resources and technologythat enables ubiquitous (often worldwide) access to data, resources,and/or technology. Cloud resources are generally characterized by greatflexibility to dynamically assign resources according to currentworkloads and needs. This can be accomplished, for example, viavirtualization, wherein resources such as hardware, storage, andnetworks are provided to a virtual machine (VM) via a softwareabstraction layer, and/or containerization, wherein instances of networkfunctions are provided in “containers” that are separated from oneanother, but that share underlying operating system, memory, and driverresources.

Deep learning algorithms, or so-called “artificial intelligence” (AI) isan important contemporary computing problem. Artificial intelligencesystems employing, for example, convolutional neural networks (CNNs) areused for contemporary computing problems such as searching large datasets, classifying documents, computer vision, self-operating machinery(including self-driving cars), and many others.

Such deep learning algorithms often involve a multistage process,including a training phase wherein the artificial intelligence model istrained with a pre-seeded data set, and an operational phase wherein itperforms its work. The training phase need not be a discrete phase thatoccurs before the operational phase, but in some cases can be acontinuous process in which the CNN continues to receive feedback andadditional training data so that it can further refine its processes.The CNN itself is often employed in a highly distributed fashion, suchas in a high-performance computing (HPC) cluster, or in a large datacenter, such as may be operated by a cloud service provider (CSP). Alarge number of compute nodes can work on the artificial intelligenceproblem in parallel, and thus produce valuable results in useful timeperiods.

Operators of such neural networks may need to run very large-scaletraining tasks for their deep learning problem sets. In some cases, ahardware acceleration solution (such as Intel® Lake Crest) may be usedto provide acceleration of the training problem. The hardwareaccelerator could include a coprocessor, a field-programmable gate array(FPGA), an application-specific integrated circuit (ASIC), a graphicsprocessing unit (GPU), firmware, or other accelerated computing devices.The operator of the neural network may need to be able to bring togethermany accelerators in a flexible and transparent way so thatcommunication among the accelerators does not become cumbersome for thesolution itself.

Some existing systems use scale-out AI accelerators, such as GPUs. Inthese architectures, coordination between accelerators may be done atthe software level. Data may need to be given to accelerators, theaccelerators compute their results, and then results are moved fromaccelerators into system memory, and from there are distributed amongdifferent scale-out nodes, and then to the accelerators attached tothose nodes. Thus, for the various accelerators in an AI system tocommunicate with one another, there may be multiple levels of passingdata from accelerator to processor, from processor to processor, andthen from processor to another accelerator. This can include cacheoperations, main memory accesses, and other memory operations that cansubstantially slow the operation of the accelerators. Furthermore, thesoftware involved in moving these data around introduces overhead to theprocess. One or more cores may need to be dedicated simply to runningthe software for moving data between the various accelerators.

This can lead to substantial overhead and waste of compute resources.For example, sources of overhead may include:

1. Control and data plane in fabric software.

2. Movement of data within different memory tiers.

3. Coordination of scale-out instances performed in software.

This overhead can become a limiting factor for the overall performanceof the AI system as the deep learning training models are spread acrossmany nodes, including cores and accelerator devices. The overhead alsotaxes the limited power and thermal capacities of each node, becauseCPUs and accelerator devices both need to keep running to perform thedeep learning training and the communication tasks that flow from thetraining.

Some existing solutions provide optimizations that are conscious ofsystem configuration to reduce the communication and coordination thatis necessary in the deep learning training system. For example, thesemay strive to reduce communication, even sometimes at the expense ofdata parallelism. Some software solutions may also focus on backgroundmemory management, which controls the movement of data between CPUs andGPUs, thereby optimizing execution performance. Other solutionsintroduce parameter servers to simplify the data exchange andsynchronization steps between the different nodes. These may be read andwritten by all applications during a coordination step. In thisapproach, performance of training becomes dependent on the parameterserver implementation.

In these cases, coordination in scale-out approaches is limited byscalability. Furthermore, configuration-conscious optimizations areerror-prone, difficult to validate, and may require trial and error,ultimately leading to compromises or trade-offs that need to berevisited as problem sizes and system configurations change.Furthermore, while these solutions provide some benefits, they do notavoid data transfer between different memory tiers.

In some cases, these existing solutions also do not address thecommunication difficulties inherent in the problem. For example, it maybe difficult to employ these approaches in the context of edge cloudcomputing appliances, where the resources available to the tenants(e.g., in the form of regular computing cores) are limited. Thus, theoverhead factors are difficult to mitigate by mixing hardwareaccelerators (such as Intel® Lake Crest) with existing compute systemssuch as ordinary x86 processors. This can create multitenancy challengesas well, since what each tenant has available to it is limited. Thus,while some nodes may have surplus acceleration capacity, others may lackthe necessary CPU cycles available to the tenant so that the tenant canbenefit from tapping that capacity. Ultimately, such software stackmediated communications are vulnerable to “siloing.”

It is possible, however, to address the limitations of software stackmediated communications between accelerators by extending theaccelerators themselves. Embodiments of the present specification extendthe accelerator inter-chip link (ICL), for example the Lake Crest ICL,which provides the communication logic between AI appliances within aplatform. These extensions may provide novel logic for transparentconnections with an inter-device communication protocol at data centeror multiple data center-level connected appliances via a secure widearea network (WAN). Transparency of communication may be achieved byprogramming the routing and forwarding into a novel system decoderelement that may be provided on the accelerator itself. The systemdecoder may be provided in any appropriate form. For example, if theaccelerator is a GPU, the system decoder may be provided in a read-onlymemory (ROM) or firmware, or may be provided as a coprocessor, ASIC,FPGA, or other logic device that provides the system decoder functionsdescribed in this specification. The system decoder described hereinachieves communication between devices within the same physical chassis(which may be referred to as a “drawer” or “capsule”) via a platformICL, and further provides a new inter-platform fabric ICL that isconfigured to natively tunnel inter-device protocol traffic via aninter-chassis fabric, such as Intel® Omni-Path, Ethernet, or some otherdata center fabric. Thus, the transparency of communication extendsseamlessly to other devices that are situated in different platformunits such as different racks, drawers, capsules, or even in differentdata centers.

This enables establishing a connection between any grouping of AIappliances that are joined together, while abstracting out the detailssuch as what type of physical and network channel exists between them,so that computational logic in the appliances can work seamlessly as ifoperating within an arbitrarily large, scaled-up instance as necessary.It may also operate within scaled-down or disaggregated instances, whenthe need goes away for high-bandwidth inter-device communication. Arelatively small amount of programmable logic (such as in an FPGA) mayallow extensible support for multiple communication protocols to makethe architecture easier to evolve and to keep it easily immersibleacross different data centers, clouds, and fog designs.

Advantageously, the system decoder of the present specification enablestransparent coordination between accelerators without having to transitdata flows between them through other memory tiers, such as up throughmain memory, and without requiring CPUs to intervene to performcross-platform transfers. This enables faster synchronization,streamlined data exchanges, and faster execution times on training datasets. It also provides advantages relative to existing high-speed linksbetween GPUs or FPGAs. Unlike those existing high-speed links, whichoperate within a single physical platform, the system agent of thepresent specification provides for scalable compositions of distributedacceleration assets and scaled-up and scaled-down deployments on adynamic basis across multiple drawers, modules, racks, or even datacenters.

The system decoder described herein provides beneficial capabilitiesthat permit dynamic composition of devices in an AI application with lowlatency sharing of data among the devices. The system decoder extendsthe current architecture to achieve elastic scale-up and scale-out ofavailable components to achieve high throughput and low latencypipelines or functions.

An extended ICL added to the hardware accelerator platform may includethe following components.

A system and method for a system decoder for training accelerators willnow be described with more particular reference to the attached FIGURES.It should be noted that throughout the FIGURES, certain referencenumerals may be repeated to indicate that a particular device or blockis wholly or substantially consistent across the FIGURES. This is not,however, intended to imply any particular relationship between thevarious embodiments disclosed. In certain examples, a genus of elementsmay be referred to by a particular reference numeral (“widget 10”),while individual species or examples of the genus may be referred to bya hyphenated numeral (“first specific widget 10-1” and “second specificwidget 10-2”).

FIG. 1 is a block diagram of selected components of a data center withconnectivity to network 100 of a cloud service provider (CSP) 102,according to one or more examples of the present specification. CSP 102may be, by way of nonlimiting example, a traditional enterprise datacenter, an enterprise “private cloud,” or a “public cloud,” providingservices such as infrastructure as a service (IaaS), platform as aservice (PaaS), or software as a service (SaaS).

CSP 102 may provision some number of workload clusters 118, which may beclusters of individual servers, blade servers, rackmount servers, or anyother suitable server topology. In this illustrative example, twoworkload clusters, 118-1 and 118-2 are shown, each providing rackmountservers 146 in a chassis 148.

In this illustration, workload clusters 118 are shown as modularworkload clusters conforming to the rack unit (“U”) standard, in which astandard rack, 19 inches wide, may be built to accommodate 42 units(42U), each 1.75 inches high and approximately 36 inches deep. In thiscase, compute resources such as processors, memory, storage,accelerators, and switches may fit into some multiple of rack units fromone to 42.

Each server 146 may host a standalone operating system and provide aserver function, or servers may be virtualized, in which case they maybe under the control of a virtual machine manager (VMM), hypervisor,and/or orchestrator, and may host one or more virtual machines, virtualservers, or virtual appliances. These server racks may be collocated ina single data center, or may be located in different geographic datacenters. Depending on the contractual agreements, some servers 146 maybe specifically dedicated to certain enterprise clients or tenants,while others may be shared.

The various devices in a data center may be connected to each other viaa switching fabric 170, which may include one or more high speed routingand/or switching devices. Switching fabric 170 may provide both“north-south” traffic (e.g., traffic to and from the wide area network(WAN), such as the internet), and “east-west” traffic (e.g., trafficacross the data center). Historically, north-south traffic accounted forthe bulk of network traffic, but as web services become more complex anddistributed, the volume of east-west traffic has risen. In many datacenters, east-west traffic now accounts for the majority of traffic.

Furthermore, as the capability of each server 146 increases, trafficvolume may further increase. For example, each server 146 may providemultiple processor slots, with each slot accommodating a processorhaving four to eight cores, along with sufficient memory for the cores.Thus, each server may host a number of VMs, each generating its owntraffic.

To accommodate the large volume of traffic in a data center, a highlycapable switching fabric 170 may be provided. Switching fabric 170 isillustrated in this example as a “flat” network, wherein each server 146may have a direct connection to a top-of-rack (ToR) switch 120 (e.g., a“star” configuration), and each ToR switch 120 may couple to a coreswitch 130. This two-tier flat network architecture is shown only as anillustrative example. In other examples, other architectures may beused, such as three-tier star or leaf-spine (also called “fat tree”topologies) based on the “Clos” architecture, hub-and-spoke topologies,mesh topologies, ring topologies, or 3-D mesh topologies, by way ofnonlimiting example.

The fabric itself may be provided by any suitable interconnect. Forexample, each server 146 may include an Intel® Host Fabric Interface(HFI), a network interface card (NIC), or other host interface. The hostinterface itself may couple to one or more processors via aninterconnect or bus, such as PCI, PCIe, or similar, and in some cases,this interconnect bus may be considered to be part of fabric 170.

The interconnect technology may be provided by a single interconnect ora hybrid interconnect, such as where PCIe provides on-chipcommunication, 1 Gb or 10 Gb copper Ethernet provides relatively shortconnections to a ToR switch 120, and optical cabling provides relativelylonger connections to core switch 130. Interconnect technologiesinclude, by way of nonlimiting example, Intel® Omni-Path™, TrueScale™,Ultra Path Interconnect (UPI) (formerly called QPI or KTI),FibreChannel, Ethernet, FibreChannel over Ethernet (FCoE), InfiniBand,PCI, PCIe, or fiber optics, to name just a few. Some of these will bemore suitable for certain deployments or functions than others, andselecting an appropriate fabric for the instant application is anexercise of ordinary skill.

Note however that while high-end fabrics such as Omni-Path™ are providedherein by way of illustration, more generally, fabric 170 may be anysuitable interconnect or bus for the particular application. This could,in some cases, include legacy interconnects like local area networks(LANs), token ring networks, synchronous optical networks (SONET),asynchronous transfer mode (ATM) networks, wireless networks such asWiFi and Bluetooth, “plain old telephone system” (POTS) interconnects,or similar. It is also expressly anticipated that in the future, newnetwork technologies will arise to supplement or replace some of thoselisted here, and any such future network topologies and technologies canbe or form a part of fabric 170.

In certain embodiments, fabric 170 may provide communication services onvarious “layers,” as originally outlined in the OSI seven-layer networkmodel. In contemporary practice, the OSI model is not followed strictly.In general terms, layers 1 and 2 are often called the “Ethernet” layer(though in large data centers, Ethernet has often been supplanted bynewer technologies). Layers 3 and 4 are often referred to as thetransmission control protocol/internet protocol (TCP/IP) layer (whichmay be further subdivided into TCP and IP layers). Layers 5-7 may bereferred to as the “application layer.” These layer definitions aredisclosed as a useful framework, but are intended to be nonlimiting.

FIG. 2 is a block diagram of an end-user computing device 200, accordingto one or more examples of the present specification.

In this example, a fabric 270 is provided to interconnect variousaspects of computing device 200. Fabric 270 may be the same as fabric170 of FIG. 1, or may be a different fabric. As above, fabric 270 may beprovided by any suitable interconnect technology. In this example,Intel® Omni-Path™ is used as an illustrative and nonlimiting example.

As illustrated, computing device 200 includes a number of logic elementsforming a plurality of nodes. It should be understood that each node maybe provided by a physical server, a group of servers, or other hardware.Each server may be running one or more virtual machines as appropriateto its application.

Node 0 208 is a processing node including a processor socket 0 andprocessor socket 1. The processors may be, for example, Intel® Xeon™processors with a plurality of cores, such as 4 or 8 cores. Node 0 208may be configured to provide network or workload functions, such as byhosting a plurality of virtual machines or virtual appliances.

Onboard communication between processor socket 0 and processor socket 1may be provided by an onboard uplink 278. This may provide a very highspeed, short-length interconnect between the two processor sockets, sothat virtual machines running on node 0 208 can communicate with oneanother at very high speeds. To facilitate this communication, a virtualswitch (vSwitch) may be provisioned on node 0 208, which may beconsidered to be part of fabric 270.

Node 0 208 connects to fabric 270 via an HFI 272. HFI 272 may connect toan Intel® Omni-Path™ fabric. In some examples, communication with fabric270 may be tunneled, such as by providing UPI tunneling over Omni-Path™.

Because computing device 200 may provide many functions in a distributedfashion that in previous generations were provided onboard, a highlycapable HFI 272 may be provided. HFI 272 may operate at speeds ofmultiple gigabits per second, and in some cases may be tightly coupledwith node 0 208. For example, in some embodiments, the logic for HFI 272is integrated directly with the processors on a system-on-a-chip. Thisprovides very high speed communication between HFI 272 and the processorsockets, without the need for intermediary bus devices, which mayintroduce additional latency into the fabric. However, this is not toimply that embodiments where HFI 272 is provided over a traditional busare to be excluded. Rather, it is expressly anticipated that in someexamples, HFI 272 may be provided on a bus, such as a PCIe bus, which isa serialized version of PCI that provides higher speeds than traditionalPCI. Throughout computing device 200, various nodes may providedifferent types of HFIs 272, such as onboard HFIs and plug-in HFIs. Itshould also be noted that certain blocks in a system on a chip may beprovided as intellectual property (IP) blocks that can be “dropped” intoan integrated circuit as a modular unit. Thus, HFI 272 may in some casesbe derived from such an IP block.

Note that in “the network is the device” fashion, node 0 208 may providelimited or no onboard memory or storage. Rather, node 0 208 may relyprimarily on distributed services, such as a memory server and anetworked storage server. Onboard, node 0 208 may provide onlysufficient memory and storage to bootstrap the device and get itcommunicating with fabric 270. This kind of distributed architecture ispossible because of the very high speeds of contemporary data centers,and may be advantageous because there is no need to over-provisionresources for each node. Rather, a large pool of high-speed orspecialized memory may be dynamically provisioned between a number ofnodes, so that each node has access to a large pool of resources, butthose resources do not sit idle when that particular node does not needthem.

In this example, a node 1 memory server 204 and a node 2 storage server210 provide the operational memory and storage capabilities of node 0208. For example, memory server node 1 204 may provide remote directmemory access (RDMA), whereby node 0 208 may access memory resources onnode 1 204 via fabric 270 in a DMA fashion, similar to how it wouldaccess its own onboard memory. The memory provided by memory server 204may be traditional memory, such as double data rate type 3 (DDR3)dynamic random access memory (DRAM), which is volatile, or may be a moreexotic type of memory, such as a persistent fast memory (PFM) likeIntel® 3D Crosspoint™ (3DXP), which operates at DRAM-like speeds, but isnonvolatile.

Similarly, rather than providing an onboard hard disk for node 0 208, astorage server node 2 210 may be provided. Storage server 210 mayprovide a networked bunch of disks (NBOD), PFM, redundant array ofindependent disks (RAID), redundant array of independent nodes (RAIN),network attached storage (NAS), optical storage, tape drives, or othernonvolatile memory solutions.

Thus, in performing its designated function, node 0 208 may accessmemory from memory server 204 and store results on storage provided bystorage server 210. Each of these devices couples to fabric 270 via anHFI 272, which provides fast communication that makes these technologiespossible.

By way of further illustration, node 3 206 is also depicted. Node 3 206also includes an HFI 272, along with two processor sockets internallyconnected by an uplink. However, unlike node 0 208, node 3 206 includesits own onboard memory 222 and storage 250. Thus, node 3 206 may beconfigured to perform its functions primarily onboard, and may not berequired to rely upon memory server 204 and storage server 210. However,in appropriate circumstances, node 3 206 may supplement its own onboardmemory 222 and storage 250 with distributed resources similar to node 0208.

Computing device 200 may also include accelerators 230. These mayprovide various accelerated functions, including hardware or coprocessoracceleration for functions such as packet processing, encryption,decryption, compression, decompression, network security, or otheraccelerated functions in the data center. In some examples, accelerators230 may include deep learning accelerators that may be directly attachedto one or more cores in nodes such as node 0 208 or node 3 206. Examplesof such accelerators can include, by way of nonlimiting example, Intel®QuickData Technology (QDT), Intel® QuickAssist Technology (QAT), Intel®Direct Cache Access (DCA), Intel® Extended Message Signaled Interrupt(MSI-X), Intel® Receive Side Coalescing (RSC), and other accelerationtechnologies.

The basic building block of the various components disclosed herein maybe referred to as “logic elements.” Logic elements may include hardware(including, for example, a software-programmable processor, an ASIC, oran FPGA), external hardware (digital, analog, or mixed-signal),software, reciprocating software, services, drivers, interfaces,components, modules, algorithms, sensors, components, firmware,microcode, programmable logic, or objects that can coordinate to achievea logical operation. Furthermore, some logic elements are provided by atangible, non-transitory computer-readable medium having stored thereonexecutable instructions for instructing a processor to perform a certaintask. Such a non-transitory medium could include, for example, a harddisk, solid state memory or disk, read-only memory (ROM), persistentfast memory (PFM) (e.g., Intel® 3D Crosspoint™), external storage,redundant array of independent disks (RAID), redundant array ofindependent nodes (RAIN), network-attached storage (NAS), opticalstorage, tape drive, backup system, cloud storage, or any combination ofthe foregoing by way of nonlimiting example. Such a medium could alsoinclude instructions programmed into an FPGA, or encoded in hardware onan ASIC or processor.

FIG. 3 illustrates a block diagram of components of a computing platform302A according to one or more examples of the present specification. Inthe embodiment depicted, platforms 302A, 302B, and 302C, along with adata center management platform 306 and data analytics engine 304 areinterconnected via network 308. In other embodiments, a computer systemmay include any suitable number of (i.e., one or more) platforms. Insome embodiments (e.g., when a computer system only includes a singleplatform), all or a portion of the system management platform 306 may beincluded on a platform 302. A platform 302 may include platform logic310 with one or more central processing units (CPUs) 312, memories 314(which may include any number of different modules), chipsets 316,communication interfaces 318, and any other suitable hardware and/orsoftware to execute a hypervisor 320 or other operating system capableof executing workloads associated with applications running on platform302. In some embodiments, a platform 302 may function as a host platformfor one or more guest systems 322 that invoke these applications.Platform 302A may represent any suitable computing environment, such asa high performance computing environment, a data center, acommunications service provider infrastructure (e.g., one or moreportions of an Evolved Packet Core), an in-memory computing environment,a computing system of a vehicle (e.g., an automobile or airplane), anInternet of Things environment, an industrial control system, othercomputing environment, or combination thereof.

In various embodiments of the present disclosure, accumulated stressand/or rates of stress accumulated of a plurality of hardware resources(e.g., cores and uncores) are monitored and entities (e.g., systemmanagement platform 306, hypervisor 320, or other operating system) ofcomputer platform 302A may assign hardware resources of platform logic310 to perform workloads in accordance with the stress information. Insome embodiments, self-diagnostic capabilities may be combined with thestress monitoring to more accurately determine the health of thehardware resources. Each platform 302 may include platform logic 310.Platform logic 310 comprises, among other logic enabling thefunctionality of platform 302, one or more CPUs 312, memory 314, one ormore chipsets 316, and communication interfaces 328. Although threeplatforms are illustrated, computer platform 302A may be interconnectedwith any suitable number of platforms. In various embodiments, aplatform 302 may reside on a circuit board that is installed in achassis, rack, or other suitable structure that comprises multipleplatforms coupled together through network 308 (which may comprise,e.g., a rack or backplane switch).

CPUs 312 may each comprise any suitable number of processor cores andsupporting logic (e.g., uncores). The cores may be coupled to eachother, to memory 314, to at least one chipset 316, and/or to acommunication interface 318, through one or more controllers residing onCPU 312 and/or chipset 316. In particular embodiments, a CPU 312 isembodied within a socket that is permanently or removably coupled toplatform 302A. Although four CPUs are shown, a platform 302 may includeany suitable number of CPUs.

Memory 314 may comprise any form of volatile or nonvolatile memoryincluding, without limitation, magnetic media (e.g., one or more tapedrives), optical media, random access memory (RAM), read-only memory(ROM), flash memory, removable media, or any other suitable local orremote memory component or components. Memory 314 may be used for short,medium, and/or long term storage by platform 302A. Memory 314 may storeany suitable data or information utilized by platform logic 310,including software embedded in a computer readable medium, and/orencoded logic incorporated in hardware or otherwise stored (e.g.,firmware). Memory 314 may store data that is used by cores of CPUs 312.In some embodiments, memory 314 may also comprise storage forinstructions that may be executed by the cores of CPUs 312 or otherprocessing elements (e.g., logic resident on chipsets 316) to providefunctionality associated with the manageability engine 326 or othercomponents of platform logic 310. A platform 302 may also include one ormore chipsets 316 comprising any suitable logic to support the operationof the CPUs 312. In various embodiments, chipset 316 may reside on thesame die or package as a CPU 312 or on one or more different dies orpackages. Each chipset may support any suitable number of CPUs 312. Achipset 316 may also include one or more controllers to couple othercomponents of platform logic 310 (e.g., communication interface 318 ormemory 314) to one or more CPUs. In the embodiment depicted, eachchipset 316 also includes a manageability engine 326. Manageabilityengine 326 may include any suitable logic to support the operation ofchipset 316. In a particular embodiment, a manageability engine 326(which may also be referred to as an innovation engine) is capable ofcollecting real-time telemetry data from the chipset 316, the CPU(s) 312and/or memory 314 managed by the chipset 316, other components ofplatform logic 310, and/or various connections between components ofplatform logic 310. In various embodiments, the telemetry data collectedincludes the stress information described herein.

In various embodiments, a manageability engine 326 operates as anout-of-band asynchronous compute agent which is capable of interfacingwith the various elements of platform logic 310 to collect telemetrydata with no or minimal disruption to running processes on CPUs 312. Forexample, manageability engine 326 may comprise a dedicated processingelement (e.g., a processor, controller, or other logic) on chipset 316,which provides the functionality of manageability engine 326 (e.g., byexecuting software instructions), thus conserving processing cycles ofCPUs 312 for operations associated with the workloads performed by theplatform logic 310. Moreover the dedicated logic for the manageabilityengine 326 may operate asynchronously with respect to the CPUs 312 andmay gather at least some of the telemetry data without increasing theload on the CPUs.

A manageability engine 326 may process telemetry data it collects(specific examples of the processing of stress information will beprovided herein). In various embodiments, manageability engine 326reports the data it collects and/or the results of its processing toother elements in the computer system, such as one or more hypervisors320 or other operating systems and/or system management software (whichmay run on any suitable logic such as system management platform 306).In particular embodiments, a critical event such as a core that hasaccumulated an excessive amount of stress may be reported prior to thenormal interval for reporting telemetry data (e.g., a notification maybe sent immediately upon detection).

Additionally, manageability engine 326 may include programmable codeconfigurable to set which CPU(s) 312 a particular chipset 316 willmanage and/or which telemetry data will be collected.

Chipsets 316 also each include a communication interface 328.Communication interface 328 may be used for the communication ofsignaling and/or data between chipset 316 and one or more I/O devices,one or more networks 308, and/or one or more devices coupled to network308 (e.g., system management platform 306). For example, communicationinterface 328 may be used to send and receive network traffic such asdata packets. In a particular embodiment, a communication interface 328comprises one or more physical network interface controllers (NICs),also known as network interface cards or network adapters. A NIC mayinclude electronic circuitry to communicate using any suitable physicallayer and data link layer standard such as Ethernet (e.g., as defined bya IEEE 802.3 standard), Fibre Channel, InfiniBand, Wi-Fi, or othersuitable standard. A NIC may include one or more physical ports that maycouple to a cable (e.g., an Ethernet cable). A NIC may enablecommunication between any suitable element of chipset 316 (e.g.,manageability engine 326 or switch 330) and another device coupled tonetwork 308. In various embodiments a NIC may be integrated with thechipset (i.e., may be on the same integrated circuit or circuit board asthe rest of the chipset logic) or may be on a different integratedcircuit or circuit board that is electromechanically coupled to thechipset.

In particular embodiments, communication interfaces 328 may allowcommunication of data (e.g., between the manageability engine 326 andthe data center management platform 306) associated with management andmonitoring functions performed by manageability engine 326. In variousembodiments, manageability engine 326 may utilize elements (e.g., one ormore NICs) of communication interfaces 328 to report the telemetry data(e.g., to system management platform 306) in order to reserve usage ofNICs of communication interface 318 for operations associated withworkloads performed by platform logic 310.

Switches 330 may couple to various ports (e.g., provided by NICs) ofcommunication interface 328 and may switch data between these ports andvarious components of chipset 316 (e.g., one or more PeripheralComponent Interconnect Express (PCIe) lanes coupled to CPUs 312).Switches 330 may be a physical or virtual (i.e., software) switch.

Platform logic 310 may include an additional communication interface318. Similar to communication interfaces 328, communication interfaces318 may be used for the communication of signaling and/or data betweenplatform logic 310 and one or more networks 308 and one or more devicescoupled to the network 308. For example, communication interface 318 maybe used to send and receive network traffic such as data packets. In aparticular embodiment, communication interfaces 318 comprise one or morephysical NICs. These NICs may enable communication between any suitableelement of platform logic 310 (e.g., CPUs 312 or memory 314) and anotherdevice coupled to network 308 (e.g., elements of other platforms orremote computing devices coupled to network 308 through one or morenetworks).

Platform logic 310 may receive and perform any suitable types ofworkloads. A workload may include any request to utilize one or moreresources of platform logic 310, such as one or more cores or associatedlogic. For example, a workload may comprise a request to instantiate asoftware component, such as an I/O device driver 324 or guest system322; a request to process a network packet received from a virtualmachine 332 or device external to platform 302A (such as a network nodecoupled to network 308); a request to execute a process or threadassociated with a guest system 322, an application running on platform302A, a hypervisor 320 or other operating system running on platform302A; or other suitable processing request.

A virtual machine 332 may emulate a computer system with its owndedicated hardware. A virtual machine 332 may run a guest operatingsystem on top of the hypervisor 320. The components of platform logic310 (e.g., CPUs 312, memory 314, chipset 316, and communicationinterface 318) may be virtualized such that it appears to the guestoperating system that the virtual machine 332 has its own dedicatedcomponents.

A virtual machine 332 may include a virtualized NIC (vNIC), which isused by the virtual machine as its network interface. A vNIC may beassigned a media access control (MAC) address or other identifier, thusallowing multiple virtual machines 332 to be individually addressable ina network.

VNF 334 may comprise a software implementation of a functional buildingblock with defined interfaces and behavior that can be deployed in avirtualized infrastructure. In particular embodiments, a VNF 334 mayinclude one or more virtual machines 332 that collectively providespecific functionalities (e.g., wide area network (WAN) optimization,virtual private network (VPN) termination, firewall operations,load-balancing operations, security functions, etc.). A VNF 334 runningon platform logic 310 may provide the same functionality as traditionalnetwork components implemented through dedicated hardware. For example,a VNF 334 may include components to perform any suitable NFV workloads,such as virtualized evolved packet core (vEPC) components, mobilitymanagement entities, 3rd Generation Partnership Project (3GPP) controland data plane components, etc.

SFC 336 is a group of VNFs 334 organized as a chain to perform a seriesof operations, such as network packet processing operations. Servicefunction chaining may provide the ability to define an ordered list ofnetwork services (e.g. firewalls, load balancers) that are stitchedtogether in the network to create a service chain.

A hypervisor 320 (also known as a virtual machine monitor) may compriselogic to create and run guest systems 322. The hypervisor 320 maypresent guest operating systems run by virtual machines with a virtualoperating platform (i.e., it appears to the virtual machines that theyare running on separate physical nodes when they are actuallyconsolidated onto a single hardware platform) and manage the executionof the guest operating systems by platform logic 310. Services ofhypervisor 320 may be provided by virtualizing in software or throughhardware assisted resources that require minimal software intervention,or both. Multiple instances of a variety of guest operating systems maybe managed by the hypervisor 320. Each platform 302 may have a separateinstantiation of a hypervisor 320.

Hypervisor 320 may be a native or bare-metal hypervisor that runsdirectly on platform logic 310 to control the platform logic and managethe guest operating systems. Alternatively, hypervisor 320 may be ahosted hypervisor that runs on a host operating system and abstracts theguest operating systems from the host operating system. Hypervisor 320may include a virtual switch 338 that may provide virtual switchingand/or routing functions to virtual machines of guest systems 322. Thevirtual switch 338 may comprise a logical switching fabric that couplesthe vNICs of the virtual machines 332 to each other, thus creating avirtual network through which virtual machines may communicate with eachother.

Virtual switch 338 may comprise a software element that is executedusing components of platform logic 310. In various embodiments,hypervisor 320 may be in communication with any suitable entity (e.g., aSDN controller) which may cause hypervisor 320 to reconfigure theparameters of virtual switch 338 in response to changing conditions inplatform 302 (e.g., the addition or deletion of virtual machines 332 oridentification of optimizations that may be made to enhance performanceof the platform).

Hypervisor 320 may also include resource allocation logic 344, which mayinclude logic for determining allocation of platform resources based onthe telemetry data (which may include stress information). Resourceallocation logic 344 may also include logic for communicating withvarious components of platform logic 310 entities of platform 302A toimplement such optimization, such as components of platform logic 310.

Any suitable logic may make one or more of these optimization decisions.For example, system management platform 306; resource allocation logic344 of hypervisor 320 or other operating system; or other logic ofcomputer platform 302A may be capable of making such decisions. Invarious embodiments, the system management platform 306 may receivetelemetry data from and manage workload placement across multipleplatforms 302. The system management platform 306 may communicate withhypervisors 320 (e.g., in an out-of-band manner) or other operatingsystems of the various platforms 302 to implement workload placementsdirected by the system management platform.

The elements of platform logic 310 may be coupled together in anysuitable manner. For example, a bus may couple any of the componentstogether. A bus may include any known interconnect, such as a multi-dropbus, a mesh interconnect, a ring interconnect, a point-to-pointinterconnect, a serial interconnect, a parallel bus, a coherent (e.g.cache coherent) bus, a layered protocol architecture, a differentialbus, or a Gunning transceiver logic (GTL) bus.

Elements of the computer platform 302A may be coupled together in anysuitable manner such as through one or more networks 308. A network 308may be any suitable network or combination of one or more networksoperating using one or more suitable networking protocols. A network mayrepresent a series of nodes, points, and interconnected communicationpaths for receiving and transmitting packets of information thatpropagate through a communication system. For example, a network mayinclude one or more firewalls, routers, switches, security appliances,antivirus servers, or other useful network devices.

FIG. 4 is a block diagram of an AI system including two platforms,namely platform 440-1 and platform 440-2.

Each platform 440 includes, for example, one or more processors 428,which may be connected to each other via a local interconnect technologysuch as Intel® QuickPath Interconnect (QPI) or similar. For example,platform 440-1 includes processor 428-1 and processor 428-2 connectedvia a QPI interconnect. Each processor 428 may also have access to itsown local resources 432. For example, processor 428-1 has access tolocal resources 432-1 and processor 428-2 has access to local resources432-2. Local resources 432 may include, for example, a network interfacecard, a host fabric interface, memory, storage, or similar. Note that inthis illustration, each processor 428 is shown as having access to itsown local resources, but it should be understood that one or more localresources 432 may be shared by processors 428. The allocation of localresources as dedicated local resources to a processor or shared localresources between multiple processors is an exercise of ordinaryengineering skill.

In this example, platform 440-1 also includes accelerators 418-1 andaccelerator 418-2. Note that in this example, each processor 428includes its own dedicated accelerator 418. For example, processor 428-1has dedicated accelerator 418-1, while processor 428-2 includesdedicated accelerator 418-2. Processors 428 may be interconnected witheach other via a known interconnect, such as PCIe. Note that PCIe is nota cache-coherent or memory-coherent interconnect. In other words, incertain existing implementations of PCIe, accelerator 418 cannotdirectly map its local memory space to the memory space of processor428. Thus, for data to be exchanged between accelerator 418 andprocessor 428, a CPU may need to operate the PCIe interconnect to movedata back and forth.

Each accelerator 418 may include, by way of nonlimiting example,accelerator hardware 420, ICL interface 416, a system decoder 412, afabric ICL 404, and a platform ICL 408. Accelerator hardware 420 may be,for example, an FPGA or a GPU, and may be programmed with a deeplearning model. As used in this specification, an accelerator hardwareprogrammed to perform a training task may be referred to as a “deeplearning module,” which may be understood to include both the hardwarethat provides the processing power, and the programming that providesthe accelerator hardware with the deep learning model. The deep learningmodel may be the same across platforms 440-1 and platform 440-2 if bothplatforms are working on the same AI problem. Platform 440-1 alsoincludes a platform control host (PCH) 424-1 that provides platformcontrol between accelerator 418-1 and accelerator 418-2.

Note that platform 440-2 is substantially similar to and possibly evenidentical to platform 440-1. Platform 440-2 includes local resources432-3 provided to processor 428-3 and local resources 432-4 provided toprocessor 428-4. Processor 428-3 and processor 428-4 may becommunicatively coupled via a local interconnect such as QPI. Processors428-3 and 428-4 also each have a respective accelerator, namelyaccelerator 418-3 and accelerator 418-4. As with platform 440-1,platform 440-2 includes a PCH 424-2.

Platform 440-1 and platform 440-2 may be coupled to each other via afast interconnect, such as Intel Omni-Path or a high-speed Ethernet orany other suitable high-speed interconnect. However, in existingsystems, there may be no mechanism for accelerators on platform 440-1 tocommunicate directly with accelerators on platform 440-2.

Embodiments of the present specification provide system decoder 412,such as system decoder 412-1 of accelerator 418-1, system decoder 412-2of accelerator 418-2, system decoder 412-3 of accelerator 418-3, andsystem decoder 412-4 of accelerator 418-4. System decoders 412 areprovided to route data from one training model instance to another. Inother words, system decoder 412 defines paths among the variousaccelerators 418 and routes data through them as they arrive in adevice.

When a training model is started within accelerator hardware 420 of anaccelerator 418, new training instances in the different accelerators418 are created. The system decoder 412 of each accelerator 418 may beprogrammed with the necessary information about which model is scheduledon which accelerator 418. Thus, each system decoder 412 knows whichmodels are running on which hardware device so that it can make routingdecisions. System decoder 412 may also use, for example, a small tableto perform lookups. It could also maintain a table in system memory or aportion thereof that is accessible to it. The system decoder may alsoplace routing information as header data on a packet to ensure thatsystem decoders on other accelerators 418 forward the dataappropriately.

Platform ICL 408 implements a link protocol between differentaccelerator devices within the same platform. This can be accomplishedwithout the necessity of communicating via the data center fabric. Forexample, operating platform ICL 408-1, accelerator 418-1 can communicatedirectly with accelerator 418-2 via platform ICL 408-2. However,accelerator 418-1 may not be able to communicate directly withaccelerator 418-3 or accelerator 418-4 via platform ICL 408. Systemdecoder 412 may operate platform ICLs 408 when moving data to aparticular model training instance from one accelerator to anotherwithin the same chassis or capsule, in this case within platform 440-1or within platform 440-2.

Fabric ICL 404 provides communication between accelerators 418 ondifferent platforms 440. Fabric ICL 404 may be, for example, an FPGA orother accelerated hardware. Fabric ICL 404 is used by system decoder 412and implements the protocol for use when communication is betweenacceleration devices hosted in different platforms 440. In FIG. 4, forexample, fabric ICL 404 is operated in a computation that needs to occurin accelerator 418-1 on platform 440-1 with data sent to accelerator418-4 on platform 440-2. Because the actual communication occurs via theinter-platform fabric that is also used by the CPU and other platformcomponents, the communication protocol for over-the-fabriccommunications between accelerators 418 may be implemented inprogrammable logic such as an FPGA. Fabric ICL 404 may provide tunnelingof packets destined for other accelerators 418 on different platforms440.

Note that the actual order of communications may be based on thespecifics of the configuration. For example, in some existing systems,accelerators 418 communicate in a round-robin fashion, in which case, ifthere are multiple accelerators 418 on a platform 440, then oneaccelerator 418 may be designated by the system decoder 412 as a“gateway” device to other accelerators on the platform 440. Note that inthis illustration, for simplicity, only two processors 428 with oneaccelerator 418 each are shown on each platform 440. However, in realworld deployments, there may be many processors with many accelerators,so that communications can be more complicated.

In one example, accelerator 418-2 may generate data that need to bepassed to accelerator 418-4. In this case, accelerators 418-1 and 418-3may be designated as the “gateway” devices for their respectiveplatforms 440. Thus, system decoder 412-2 does not operate its fabricICL 404-2 to communicate directly with fabric ICL 404-4 of accelerator418-4. Rather, system decoder 412-2 may first operate platform ICL 408-2to pass the data to accelerator 418-1 via platform ICL 408-1. Systemdecoder 412-1 may then operate fabric ICL 404-1 to pass the data tofabric ICL 404-3 of accelerator 418-3. System decoder 412-3 thenoperates platform ICL 408-3 to pass the data to accelerator 418-4 viaplatform ICL 418-4.

FIG. 5 is a block diagram of a system decoder 512, according to one ormore examples of the present specification. As noted in connection withFIG. 4, a fabric ICL, such as fabric ICL 504 disclosed in thisillustration, may be an FPGA or other programmable logic such as a GPU,DSP, coprocessor, or other programmable logic device. When the system isstarted, system decoder 512 may determine the appropriate role forfabric ICL 504 within the system. Thus, system decoder 512 may need toprogram fabric ICL 504 with the appropriate routing tables, behavior,and other information to carry out the specified routing protocol. Thiscould include, for example, designating particular routes for broadcasttraffic or point-to-point traffic, by way of nonlimiting example. Asdiscussed in connection with FIG. 4, in the case of broadcast traffic,one fabric ICL 504 on the platform may be designated as the gatewaydevice for all accelerators on the platform. In the case ofpoint-to-point communications, each fabric ICL 504 on a particularaccelerator may interconnect with other fabric ICLs via the networkfabric. Fabric ICLs 504 may also connect to each other in a round-robinor mesh network configuration, or any other configuration suitable tothe needs of a particular deployment. Thus, in many instances, fabricICL 504 may benefit from having the flexibility of being adaptable tothe specific network configuration. Furthermore, fabric ICL 504 may needto be able to adapt to changing network conditions. For example, thefabric ICL 504 on a particular platform that is used as a gateway ininstances where a single fabric ICL is designated as the gateway may bedynamically allocated depending on network traffic conditions. Forexample, system decoder 512 may calculate the network cost associatedwith using different fabric ICLs 504 as the gateway device, or maycompare the network cost of using a gateway device as opposed to usingpoint-to-point or mesh connections.

System decoder 512 may have access to one or more tangible,non-transitory computer readable mediums 508, which may have storedthereon computer operable instructions to carry out the methodsdisclosed herein. This could include, by way of nonlimiting example,executable software instructions for execution on a GPU, CPU, DSP, orother programmable logic device. This could also include instructionsfor programming an FPGA with the appropriate routing information. Insome examples, this could also include instructions or masks forprogramming an ASIC with the appropriate routing.

Thus, upon system startup, or some other appropriate time, systemdecoder 512 or some other appropriate system as dictated by theembodiment may access non-transitory computer readable medium 508, andmay use information thereon to program fabric ICL 504 with theappropriate routing configuration. Note that non-transitory computerreadable medium 508 may include a number of stock configurations fromwhich system decoder 512 can select the appropriate configuration forthe present network demands, or it may include more complicatedalgorithms for crafting a highly specialized configuration for theparticular network need.

FIG. 6 is a signal flow diagram illustrating an example broadcastmessage, according to one or more examples of the present specification.In this example, four accelerator devices are illustrated, namely device610-1, device 610-2, device 610-3, and device 610-4. Device 610-1 anddevice 610-2 are both hosted on platform 612-1. Device 610-3 and 610-4are both hosted on platform 612-2.

In this illustrative embodiment, fabric ICL 604-1 is designated as thegateway fabric ICL for platform 612-1, for broadcast purposes. FabricICL 604-2 is designated as the gateway fabric ICL for platform 612-2,for broadcast purposes.

Device 610-1 may compute a datum that is to be broadcast to some or allaccelerators in the system. For purposes of this illustration, the datumcomputed by device 610-1 is to be broadcast to at least device 610-2,device 610-3, and device 610-4. Note, however, that it may be necessaryto broadcast the datum to other devices, and the embodiment illustratedherein should be understood to be nonlimiting and illustrative only.

At operation 1, the system decoder of device 610-1 first may distributethe datum to all devices on platform 612-1. Because the system decoderof device 610-1 knows that its own fabric ICL is not the gateway fabricICL for broadcast purposes on this platform, device 610-1 need take nofurther action. As part of broadcasting the datum to all devices onplatform 612-1, platform ICL 608-1 communicates the datum to otherplatform ICLs on platform 612-1, including to platform ICL 608-2.

At operation 2, the system decoder of device 610-2 determines thatfabric ICL 604-1 is the designated gateway fabric ICL for broadcastpurposes for platform 612-1. Thus, platform ICL 608-2 transfers thedatum to fabric ICL 604-1.

At operation 3, device 610-2 consumes the datum and performs any workneeded to be performed on the datum.

At operation four, because fabric ICL 604-1 is the designated gatewayfabric ICL for fabric 612-1, fabric ICL 604-1 transmits the datum tofabric ICL 604-2 of platform 612-2. Note that there may be a number ofother platforms within the AI system, and in that case, fabric ICL 604-1may broadcast the datum to a plurality of fabric ICLs on a plurality ofplatforms.

At operation 5, fabric ICL 604-2 of device 610-3 transmits the datum toplatform ICL 608-3 of device 610-3.

At operation 6, platform ICL 608-3 of device 610-3 identifies the datumas a datum to be transmitted to other devices 610 on platform 612-2.Thus, platform ICL 608-3 transmits the datum to platform ICL 608-4 ofdevice 610-4. Platform ICL 608-3 may also broadcast the datum to otherplatform ICLs 608 of other devices 610 on platform 612-2.

At operation 7, device 610-3 consumes the datum and performs any workneeded to be performed on the datum.

At operation 8, device 610-4 also consumes the datum and performs anywork that needs to be performed on the datum.

FIG. 7 is a flowchart of method 700 performed by a system decoder orother appropriate device, according to one or more examples of thepresent specification.

In block 702, the accelerator device hosting the system decoder startsup.

In block 704, the system decoder determines the necessary routingconfiguration for the platform and the device. This can include anyappropriate routing configuration, such as a mesh, point-to-pointconnections, round-robin, broadcast, or other appropriate networkconfiguration. The system decoder may interoperate and communicate withother system decoders on the platform to “elect” a gateway platform ICLin cases where a gateway is to be designated. The election of a gatewayplatform ICL can be performed according to any of the known methods forelecting a gateway or master device.

In block 708, the system decoder configures the fabric ICL according tothe routing configuration determined in block 704. In some examples,this may include reading data from a non-transitory computer readablemedium such as is illustrated in FIG. 5, and programming the fabric ICLaccordingly. This could include programming an FPGA or loading softwareto run on, for example, a coprocessor or a GPU.

In block 712, the fabric ICL determines that in block 28 there are dataavailable for use by this fabric ICL. Thus, in block 712, the fabric ICLsends the data according to the configured routing configuration. Thiscould include broadcasting the data out to all fabric ICLs on allplatforms within the system, or it could include transmitting the datato only one fabric ICL on one other device or platform.

Similarly, at block 716, the fabric ICL determines that there areincoming data 724. These data may be incoming from other devices orplatforms within the system. Thus, in block 716, the fabric ICL routesthe data according to the routing configuration. This could includetransmitting the data to the platform ICL local to the fabric ICL,and/or instructing the platform ICL to transmit the data to one or moreother devices on the platform via the platform ICL.

In block 798, the method is done.

The foregoing outlines features of one or more embodiments of thesubject matter disclosed herein. These embodiments are provided toenable a person having ordinary skill in the art (PHOSITA) to betterunderstand various aspects of the present disclosure. Certainwell-understood terms, as well as underlying technologies and/orstandards may be referenced without being described in detail. It isanticipated that the PHOSITA will possess or have access to backgroundknowledge or information in those technologies and standards sufficientto practice the teachings of the present specification.

The PHOSITA will appreciate that they may readily use the presentdisclosure as a basis for designing or modifying other processes,structures, or variations for carrying out the same purposes and/orachieving the same advantages of the embodiments introduced herein. ThePHOSITA will also recognize that such equivalent constructions do notdepart from the spirit and scope of the present disclosure, and thatthey may make various changes, substitutions, and alterations hereinwithout departing from the spirit and scope of the present disclosure.

In the foregoing description, certain aspects of some or all embodimentsare described in greater detail than is strictly necessary forpracticing the appended claims. These details are provided by way ofnon-limiting example only, for the purpose of providing context andillustration of the disclosed embodiments. Such details should not beunderstood to be required, and should not be “read into” the claims aslimitations. The phrase may refer to “an embodiment” or “embodiments.”These phrases, and any other references to embodiments, should beunderstood broadly to refer to any combination of one or moreembodiments. Furthermore, the several features disclosed in a particular“embodiment” could just as well be spread across multiple embodiments.For example, if features 1 and 2 are disclosed in “an embodiment,”embodiment A may have feature 1 but lack feature 2, while embodiment Bmay have feature 2 but lack feature 1.

This specification may provide illustrations in a block diagram format,wherein certain features are disclosed in separate blocks. These shouldbe understood broadly to disclose how various features interoperate, butare not intended to imply that those features must necessarily beembodied in separate hardware or software. Furthermore, where a singleblock discloses more than one feature in the same block, those featuresneed not necessarily be embodied in the same hardware and/or software.For example, a computer “memory” could in some circumstances bedistributed or mapped between multiple levels of cache or local memory,main memory, battery-backed volatile memory, and various forms ofpersistent memory such as a hard disk, storage server, optical disk,tape drive, or similar.

In certain embodiments, some of the components may be omitted orconsolidated. In a general sense, the arrangements depicted in thefigures may be more logical in their representations, whereas a physicalarchitecture may include various permutations, combinations, and/orhybrids of these elements. Countless possible design configurations canbe used to achieve the operational objectives outlined herein.Accordingly, the associated infrastructure has a myriad of substitutearrangements, design choices, device possibilities, hardwareconfigurations, software implementations, and equipment options.

References may be made herein to a computer-readable medium, which maybe a tangible and non-transitory computer-readable medium. As used inthis specification and throughout the claims, a “computer-readablemedium” should be understood to include one or more computer-readablemediums of the same or different types. A computer-readable medium mayinclude, by way of non-limiting example, an optical drive (e.g.,CD/DVD/Blu-Ray), a hard drive, a solid-state drive, a flash memory, orother non-volatile medium. A computer-readable medium could also includea medium such as a read-only memory (ROM), an FPGA or ASIC configured tocarry out the desired instructions, stored instructions for programmingan FPGA or ASIC to carry out the desired instructions, an intellectualproperty (IP) block that can be integrated in hardware into othercircuits, or instructions encoded directly into hardware or microcode ona processor such as a microprocessor, digital signal processor (DSP),microcontroller, or in any other suitable component, device, element, orobject where appropriate and based on particular needs. A nontransitorystorage medium herein is expressly intended to include any nontransitoryspecial-purpose or programmable hardware configured to provide thedisclosed operations, or to cause a processor to perform the disclosedoperations.

Various elements may be “communicatively,” “electrically,”“mechanically,” or otherwise “coupled” to one another throughout thisspecification and the claims. Such coupling may be a direct,point-to-point coupling, or may include intermediary devices. Forexample, two devices may be communicatively coupled to one another via acontroller that facilitates the communication. Devices may beelectrically coupled to one another via intermediary devices such assignal boosters, voltage dividers, or buffers. Mechanically-coupleddevices may be indirectly mechanically coupled.

Any “module” or “engine” disclosed herein may refer to or includesoftware, a software stack, a combination of hardware, firmware, and/orsoftware, a circuit configured to carry out the function of the engineor module, or any computer-readable medium as disclosed above. Suchmodules or engines may, in appropriate circumstances, be provided on orin conjunction with a hardware platform, which may include hardwarecompute resources such as a processor, memory, storage, interconnects,networks and network interfaces, accelerators, or other suitablehardware. Such a hardware platform may be provided as a singlemonolithic device (e.g., in a PC form factor), or with some or part ofthe function being distributed (e.g., a “composite node” in a high-enddata center, where compute, memory, storage, and other resources may bedynamically allocated and need not be local to one another).

There may be disclosed herein flow charts, signal flow diagram, or otherillustrations showing operations being performed in a particular order.Unless otherwise expressly noted, or unless required in a particularcontext, the order should be understood to be a non-limiting exampleonly. Furthermore, in cases where one operation is shown to followanother, other intervening operations may also occur, which may berelated or unrelated. Some operations may also be performedsimultaneously or in parallel. In cases where an operation is said to be“based on” or “according to” another item or operation, this should beunderstood to imply that the operation is based at least partly on oraccording at least partly to the other item or operation. This shouldnot be construed to imply that the operation is based solely orexclusively on, or solely or exclusively according to the item oroperation.

All or part of any hardware element disclosed herein may readily beprovided in a system-on-a-chip (SoC), including a central processingunit (CPU) package. An SoC represents an integrated circuit (IC) thatintegrates components of a computer or other electronic system into asingle chip. Thus, for example, client devices or server devices may beprovided, in whole or in part, in an SoC. The SoC may contain digital,analog, mixed-signal, and radio frequency functions, all of which may beprovided on a single chip substrate. Other embodiments may include amultichip module (MCM), with a plurality of chips located within asingle electronic package and configured to interact closely with eachother through the electronic package.

In a general sense, any suitably-configured circuit or processor canexecute any type of instructions associated with the data to achieve theoperations detailed herein. Any processor disclosed herein couldtransform an element or an article (for example, data) from one state orthing to another state or thing. Furthermore, the information beingtracked, sent, received, or stored in a processor could be provided inany database, register, table, cache, queue, control list, or storagestructure, based on particular needs and implementations, all of whichcould be referenced in any suitable timeframe. Any of the memory orstorage elements disclosed herein, should be construed as beingencompassed within the broad terms “memory” and “storage,” asappropriate.

Computer program logic implementing all or part of the functionalitydescribed herein is embodied in various forms, including, but in no waylimited to, a source code form, a computer executable form, machineinstructions or microcode, programmable hardware, and variousintermediate forms (for example, forms generated by an assembler,compiler, linker, or locator). In an example, source code includes aseries of computer program instructions implemented in variousprogramming languages, such as an object code, an assembly language, ora high-level language such as OpenCL, FORTRAN, C, C++, JAVA, or HTML foruse with various operating systems or operating environments, or inhardware description languages such as Spice, Verilog, and VHDL. Thesource code may define and use various data structures and communicationmessages. The source code may be in a computer executable form (e.g.,via an interpreter), or the source code may be converted (e.g., via atranslator, assembler, or compiler) into a computer executable form, orconverted to an intermediate form such as byte code. Where appropriate,any of the foregoing may be used to build or describe appropriatediscrete or integrated circuits, whether sequential, combinatorial,state machines, or otherwise.

In one example embodiment, any number of electrical circuits of theFIGURES may be implemented on a board of an associated electronicdevice. The board can be a general circuit board that can hold variouscomponents of the internal electronic system of the electronic deviceand, further, provide connectors for other peripherals. Any suitableprocessor and memory can be suitably coupled to the board based onparticular configuration needs, processing demands, and computingdesigns.

Note that with the numerous examples provided herein, interaction may bedescribed in terms of two, three, four, or more electrical components.However, this has been done for purposes of clarity and example only. Itshould be appreciated that the system can be consolidated orreconfigured in any suitable manner. Along similar design alternatives,any of the illustrated components, modules, and elements of the FIGURESmay be combined in various possible configurations, all of which arewithin the broad scope of this specification.

Numerous other changes, substitutions, variations, alterations, andmodifications may be ascertained to one skilled in the art and it isintended that the present disclosure encompass all such changes,substitutions, variations, alterations, and modifications as fallingwithin the scope of the appended claims. In order to assist the UnitedStates Patent and Trademark Office (USPTO) and, additionally, anyreaders of any patent issued on this application in interpreting theclaims appended hereto, Applicant wishes to note that the Applicant: (a)does not intend any of the appended claims to invoke paragraph six (6)of 35 U.S.C. section 112 (pre-AIA) or paragraph (f) of the same section(post-AIA), as it exists on the date of the filing hereof unless thewords “means for” or “steps for” are specifically used in the particularclaims; and (b) does not intend, by any statement in the specification,to limit this disclosure in any way that is not otherwise expresslyreflected in the appended claims.

Example Implementations

The following examples are provided by way of illustration.

Example 1 includes an artificial intelligence (AI) system, comprising: afirst hardware platform; a fabric interface configured tocommunicatively couple the first hardware platform to a second hardwareplatform; a processor hosted on the first hardware platform andprogrammed to operate on an AI problem; and a first trainingaccelerator, comprising: an accelerator hardware; a platform inter-chiplink (ICL) configured to communicatively couple the first trainingaccelerator to a second training accelerator on the first hardwareplatform without aid of the processor; a fabric ICL to communicativelycouple the first training accelerator to a third training accelerator ona second hardware platform without aid of the processor; and a systemdecoder configured to operate the fabric ICL and platform ICL to sharedata of the accelerator hardware between the first training acceleratorand second and third training accelerators without aid of the processor.

Example 2 includes the AI system of example 1, wherein the fabric is anon-memory-coherent fabric.

Example 3 includes the AI system of example 2, wherein the non-memorycoherent fabric is a PCIe fabric.

Example 4 includes the AI system of example 1, wherein the fabric is anOmniPath fabric.

Example 5 includes the AI system of example 1, wherein the systemdecoder is configured to communicatively couple the first trainingaccelerator to the third training accelerator in a point-to-pointconfiguration.

Example 6 includes the AI system of example 1, wherein the systemdecoder is configured to communicatively couple the first trainingaccelerator to the third training accelerator in a mesh configuration.

Example 7 includes the AI system of example 1, wherein the systemdecoder is configured to designate the fabric ICL as a gateway fabricICL for the first hardware platform.

Example 8 includes the AI system of example 7, wherein the fabric ICL isto receive broadcast traffic from a single training accelerator of thesecond hardware platform.

Example 9 includes the AI system of example 7, wherein the fabric ICL isto broadcast traffic from the second training accelerator to a pluralityof hardware platforms.

Example 10 includes the AI system of any of examples 1-9, wherein thefabric ICL is an application-specific integrated circuit (ASIC).

Example 11 includes the AI system of any of examples 1-9, wherein thefabric ICL is a field-programmable gate array (FPGA).

Example 12 includes the AI system of example 11, wherein the systemdecoder comprises a non-transitory storage medium having stored thereoninstructions for configuring the fabric ICL FPGA, and wherein the systemdecoder is to program the fabric ICL FPGA with the instructions.

Example 13 includes the AI system of example 11, wherein thenon-transitory storage medium comprises different instructions toconfigure the fabric ICL for a plurality of roles, and wherein thesystem decoder is to select a role for the fabric ICL and program thefabric ICL with the instructions for that role.

Example 14 includes a system decoder for a first training accelerator tooperate on a first hardware platform, comprising: means tocommunicatively couple to a fabric interface configured tocommunicatively couple the first hardware platform to a second hardwareplatform; means to communicatively couple to an accelerator hardware;means to communicatively couple to a platform inter-chip link (ICL)configured to communicatively couple the first training accelerator to asecond training accelerator on the first hardware platform without aidof a processor; meant to communicatively couple to a fabric ICL tocommunicatively couple the first training accelerator to a thirdtraining accelerator on a second hardware platform without aid of theprocessor; and means to operate the fabric ICL and platform ICL to sharedata of the accelerator hardware between the first training acceleratorand second and third training accelerators without aid of the processor.

Example 15 includes the system decoder of example 14, wherein the fabricis a non-memory-coherent fabric.

Example 16 includes the system decoder of example 15, wherein thenon-memory coherent fabric is a PCIe fabric.

Example 17 includes the system decoder of example 14, wherein the fabricis an OmniPath fabric.

Example 18 includes the system decoder of example 14, wherein the systemdecoder is configured to communicatively couple the first trainingaccelerator to the third training accelerator in a point-to-pointconfiguration.

Example 19 includes the system decoder of example 14, wherein the systemdecoder is configured to communicatively couple the first trainingaccelerator to the third training accelerator in a mesh configuration.

Example 20 includes the system decoder of example 14, wherein the systemdecoder is configured to designate the fabric ICL as a gateway fabricICL for the first hardware platform.

Example 21 includes the system decoder of example 20, wherein the fabricICL is to receive broadcast traffic from a single training acceleratorof the second hardware platform.

Example 22 includes the system decoder of example 20, wherein the fabricICL is to broadcast traffic from the second training accelerator to aplurality of hardware platforms.

Example 23 includes the system decoder of any of examples 14-22, whereinthe fabric ICL is an application-specific integrated circuit (ASIC).

Example 24 includes the system decoder of any of examples 14-22, whereinthe fabric ICL is a field-programmable gate array (FPGA).

Example 25 includes the system decoder of example 24, wherein the systemdecoder comprises a non-transitory storage medium having stored thereoninstructions for configuring the fabric ICL FPGA, and wherein the systemdecoder is to program the fabric ICL FPGA with the instructions.

Example 26 includes the system decoder of example 24, wherein thenon-transitory storage medium comprises different instructions toconfigure the fabric ICL for a plurality of roles, and wherein thesystem decoder is to select a role for the fabric ICL and program thefabric ICL with the instructions for that role.

Example 27 includes the system decoder of any of examples 14-26, whereinthe system decoder is an intellectual property (IP) block.

Example 28 includes the system decoder of any of examples 14-26, whereinthe system decoder is a field-programmable gate array.

Example 29 includes the system decoder of any of examples 14-26, whereinthe system decoder is a processor.

Example 30 includes a method of providing communication between a firsttraining accelerator on a first hardware platform, a second trainingaccelerator on the first hardware platform, and a third trainingaccelerator on a second hardware platform, comprising: communicativelycoupling to a fabric interface configured to communicatively couple thefirst hardware platform to the second hardware platform; communicativelycoupling to an accelerator hardware of the first training accelerator;communicatively coupling to a platform inter-chip link (ICL) configuredto communicatively couple the first training accelerator to a secondtraining accelerator on the first hardware platform without aid of aprocessor; communicatively coupling to a fabric ICL to communicativelycouple the first training accelerator to a third training accelerator ona second hardware platform without aid of the processor; and operatingthe fabric ICL and platform ICL to share data between the first trainingaccelerator and second and third training accelerators without aid ofthe processor.

Example 31 includes the method of example 30, wherein the fabric is anon-memory-coherent fabric.

Example 32 includes the method of example 31, wherein the non-memorycoherent fabric is a PCIe fabric.

Example 33 includes the method of example 30, wherein the fabric is anOmniPath fabric.

Example 34 includes the method of example 30, further comprisingcommunicatively coupling the first training accelerator to the thirdtraining accelerator in a point-to-point configuration.

Example 35 includes the method of example 30, further comprisingcommunicatively coupling the first training accelerator to the thirdtraining accelerator in a mesh configuration.

Example 36 includes the method of example 30, further comprisingdesignating the fabric ICL as a gateway fabric ICL for the firsthardware platform.

Example 37 includes the method of example 30, further comprisingreceiving broadcast traffic from a single training accelerator of thesecond hardware platform.

Example 38 includes the method of example 30, further comprisingbroadcasting traffic from the second training accelerator to a pluralityof hardware platforms.

Example 39 includes a system decoder configured to perform the method ofany of examples 30-38.

Example 40 includes the system decoder example 39, wherein the fabricICL is an application-specific integrated circuit (ASIC).

Example 41 includes the system decoder of example 39, wherein the fabricICL is a field-programmable gate array (FPGA).

Example 42 includes the system decoder of example 41, wherein the systemdecoder comprises a non-transitory storage medium having stored thereoninstructions for configuring the fabric ICL FPGA, and wherein the systemdecoder is to program the fabric ICL FPGA with the instructions.

Example 43 includes the system decoder of example 41, wherein thenon-transitory storage medium comprises different instructions toconfigure the fabric ICL for a plurality of roles, and wherein thesystem decoder is to select a role for the fabric ICL and program thefabric ICL with the instructions for that role.

Example 44 includes the system decoder of any of examples 39-43, whereinthe system decoder is an intellectual property (IP) block.

Example 45 includes the system decoder of any of examples 39-43, whereinthe system decoder is a field-programmable gate array.

Example 46 includes the system decoder of any of examples 39-43, whereinthe system decoder is a processor.

1-25. (canceled)
 26. Accelerator hardware for use in association with acomputing platform, the computing platform having a physical chassis,the computing platform being for use in providing, when the computingplatform is in operation, at least one artificial intelligence(AI)-related service, the computing platform comprising multiple centralprocessing units (CPUs), multiple physical network interface controllers(NICs), and forwarding hardware, the multiple CPUs and multiple physicalNICs being comprised in multiple circuit boards communicatively coupledtogether in the computing platform, the multiple physical NICs being foruse in network traffic communication via at least one network, theaccelerator hardware comprising: at least one graphics processing unit(GPU) for being configured to execute multiple virtualized instances,the multiple virtualized instances being configurable to implement AItraining-related operations associated with at least one training modelfor use in association with the providing of the at least one AI-relatedservice; wherein: when the computing platform is in the operation, theat least one GPU is configurable to communicate, via at least onecommunication link and the forwarding hardware, with at least one otherGPU; the at least one GPU and the at least one other GPU are comprisedin the physical chassis; the at least one communication link iscomprised, at least in part, in the computing platform; when thecomputing platform is in the operation, the at least one GPU is to becommunicatively coupled via at least one other communication link to atleast one of the multiple CPUs; and the at least one communication linkand the at least one other communication link are to use respectivecommunication protocols.
 27. The accelerator hardware of claim 26,wherein: the respective communication protocols are different from eachother, at least in part.
 28. The accelerator hardware of claim 26,wherein: when the computing platform is in the operation, the at leastone GPU and the at least one other GPU are configurable to executemultiple respective virtualized instances.
 29. The accelerator hardwareof claim 28, wherein: when the computing platform is in the operation,the computing platform comprises a virtual switch for use with NICcommunications.
 30. The accelerator hardware of claim 29, wherein: theaccelerator hardware is configurable for use in deep learning-relatedoperations.
 31. The accelerator hardware of claim 30, wherein: the atleast one AI-related service is associated, at least in part, withmultitenant cloud computing.
 32. The accelerator hardware of claim 31,wherein: the at least one other communication link comprises at leastone Peripheral Component Interconnect Express link.
 33. At least onenon-transitory machine-readable storage medium storing instructions forbeing executed, at least in part, by accelerator hardware, theaccelerator hardware being for use in association with a computingplatform, the computing platform having a physical chassis, thecomputing platform being for use in providing, when the computingplatform is in operation, at least one artificial intelligence(AI)-related service, the computing platform comprising multiple centralprocessing units (CPUs), multiple physical network interface controllers(NICs), and forwarding hardware, the multiple CPUs and multiple physicalNICs being comprised in multiple circuit boards communicatively coupledtogether in the computing platform, the multiple physical NICs being foruse in network traffic communication via at least one network, theaccelerator hardware comprising at least one graphics processing unit(GPU), the instructions when executed, at least in part, by theaccelerator hardware resulting in performance of operations comprising:configuring the at least one GPU to execute multiple virtualizedinstances, the multiple virtualized instances being configurable toimplement AI training-related operations associated with at least onetraining model for use in association with the providing of the at leastone AI-related service; wherein: when the computing platform is in theoperation, the at least one GPU is configurable to communicate, via atleast one communication link and the forwarding hardware, with at leastone other GPU; the at least one GPU and the at least one other GPU arecomprised in the physical chassis; the at least one communication linkis comprised, at least in part, in the computing platform; when thecomputing platform is in the operation, the at least one GPU is to becommunicatively coupled via at least one other communication link to atleast one of the multiple CPUs; and the at least one communication linkand the at least one other communication link are to use respectivecommunication protocols.
 34. The machine-readable storage medium ofclaim 33, wherein: the respective communication protocols are differentfrom each other, at least in part.
 35. The machine-readable storagemedium of claim 33, wherein: when the computing platform is in theoperation, the at least one GPU and the at least one other GPU areconfigurable to execute multiple respective virtualized instances. 36.The machine-readable storage medium of claim 35, wherein: when thecomputing platform is in the operation, the computing platform comprisesa virtual switch for use with NIC communications.
 37. Themachine-readable storage medium of claim 36, wherein: the acceleratorhardware is configurable for use in deep learning-related operations.38. The machine-readable storage medium of claim 37, wherein: the atleast one AI-related service is associated, at least in part, withmultitenant cloud computing.
 39. The machine-readable storage medium ofclaim 38, wherein: the at least one other communication link comprisesat least one Peripheral Component Interconnect Express link.
 40. Aphysical computing platform for use in providing, when the physicalcomputing platform is in operation, at least one artificial intelligence(AI)-related service, the physical computing platform comprising:accelerator hardware; a physical chassis; multiple central processingunits (CPUs); multiple physical network interface controllers (NICs);and forwarding hardware; wherein: the multiple CPUs and multiplephysical NICs are comprised in multiple circuit boards communicativelycoupled together in the physical computing platform; the multiplephysical NICs are for use in network traffic communication via at leastone network; the accelerator hardware comprises at least one graphicsprocessing unit (GPU) for being configured to execute multiplevirtualized instances, the multiple virtualized instances beingconfigurable to implement AI training-related operations associated withat least one training model for use in association with the providing ofthe at least one AI-related service; when the physical computingplatform is in the operation, the at least one GPU is configurable tocommunicate, via at least one communication link and the forwardinghardware, with at least one other GPU; the at least one GPU and the atleast one other GPU are comprised in the physical chassis; the at leastone communication link is comprised, at least in part, in the physicalcomputing platform; when the physical computing platform is in theoperation, the at least one GPU is to be communicatively coupled via atleast one other communication link to at least one of the multiple CPUs;and the at least one communication link and the at least one othercommunication link are to use respective communication protocols. 41.The physical computing platform of claim 40, wherein: the respectivecommunication protocols are different from each other, at least in part.42. The physical computing platform of claim 40, wherein: when thephysical computing platform is in the operation, the at least one GPUand the at least one other GPU are configurable to execute multiplerespective virtualized instances.
 43. The physical computing platform ofclaim 42, wherein: when the physical computing platform is in theoperation, the physical computing platform comprises a virtual switchfor use with NIC communications.
 44. The physical computing platform ofclaim 43, wherein: the accelerator hardware is configurable for use indeep learning-related operations.
 45. The physical computing platform ofclaim 44, wherein: the at least one AI-related service is associated, atleast in part, with multitenant cloud computing.
 46. The physicalcomputing platform of claim 45, wherein: the at least one othercommunication link comprises at least one Peripheral ComponentInterconnect Express link.
 47. At least one non-transitorymachine-readable storage medium storing instructions for being executed,at least in part, by a computing platform, the computing platform beingassociated with accelerator hardware, the computing platform having aphysical chassis, the computing platform being for use in providing,when the computing platform is in operation, at least one artificialintelligence (AI)-related service, the computing platform comprisingmultiple central processing units (CPUs), multiple physical networkinterface controllers (NICs), and forwarding hardware, the multiple CPUsand multiple physical NICs being comprised in multiple circuit boardscommunicatively coupled together in the computing platform, the multiplephysical NICs being for use in network traffic communication via atleast one network, the accelerator hardware comprising at least onegraphics processing unit (GPU), the instructions when executed, at leastin part, by the computing platform resulting in performance ofoperations comprising: configuring the at least one GPU to executemultiple virtualized instances, the multiple virtualized instances beingconfigurable to implement AI training-related operations associated withat least one training model for use in association with the providing ofthe at least one AI-related service; wherein: when the computingplatform is in the operation, the at least one GPU is configurable tocommunicate, via at least one communication link and the forwardinghardware, with at least one other GPU; the at least one GPU and the atleast one other GPU are comprised in the physical chassis; the at leastone communication link is comprised, at least in part, in the computingplatform; when the computing platform is in the operation, the at leastone GPU is to be communicatively coupled via at least one othercommunication link to at least one of the multiple CPUs; and the atleast one communication link and the at least one other communicationlink are to use respective communication protocols.
 48. Themachine-readable storage medium of claim 47, wherein: the respectivecommunication protocols are different from each other, at least in part.49. The machine-readable storage medium of claim 47, wherein: when thecomputing platform is in the operation, the at least one GPU and the atleast one other GPU are configurable to execute multiple respectivevirtualized instances.
 50. The machine-readable storage medium of claim49, wherein: when the computing platform is in the operation, thecomputing platform comprises a virtual switch for use with NICcommunications.
 51. The machine-readable storage medium of claim 50,wherein: the accelerator hardware is configurable for use in deeplearning-related operations.
 52. The machine-readable storage medium ofclaim 51, wherein: the at least one AI-related service is associated, atleast in part, with multitenant cloud computing.
 53. Themachine-readable storage medium of claim 52, wherein: the at least oneother communication link comprises at least one Peripheral ComponentInterconnect Express link.